\subsection{Black-Box Testing}

The aim of black-box testing is to test whether a program plausibly solves the
problem it is supposed to solve. The focus is on the
functionality of the program from an external point of view, rather than of the
program text - which in this context is considered a black-box \cite{Test}: 2,8.
\\
The test is carried out by creating sets of inputs and
corresponding expected outputs designed in a destructive manner to reveal
potential errors in the program \cite{Test}: 8. These sets are created from an
idea of how the program might solve the problem, and they should each be
justified by testing for a unique potential error \cite{Test}: 9. We have, of
course, a very good idea about potential errors, which is reflected in the
choice of test input sets.
\\
The test data set is generally supposed to cover both
'typical' and 'extreme' input values \cite{Test}: 8. However, in regard to
searching with boolean operators, we have chosen the model of our program as the
scope of this test - that is, we only expect correct behavior given a
more or less well formatted input. The rationale behind this is that - following the 
'garbage in, garbage out' slogan - we do not want the model of our program to
handle \emph{any} input format. Rather, we would account for non-well formatted
(extreme) inputs before passing them on to the model, which we consider to be a relatively
low-risk task (i.e., it could be achieved by doing a syntactic test of the search query as
early as in the view). Due to the scope of this project, we have not prioritized
implementing this. The specifics of the limitations in the input formatting when
searching with boolean operators are described in section
\ref{boolean_limitations}.

\subsubsection{Constructing Data Files for Testing}
The construction of the test data set takes an offset in the problems our
program solves. By now, this report should have covered in some detail what
(and how) our program solves the problem of both creating searchable
data structures and performing a search within them.
\\
Both of these tasks must be exposed to extreme inputs to document the
corresponding behavior. In the case of typical inputs, we are actually testing
both tasks at once. We argue that this is fine since it is very unlikely that
two parallel errors should 'cancel each other out' during the two tasks - and
even more unlikely (growing exponentially) that this should happen
across several queries. The same argument goes for testing with page ranking
enabled an error in which is very unlikely to cancel out errors across all
tests. Besides, enabling page ranking presents the obvious advantage of testing the
sorting algorithm as well as presenting the test outcomes in a more readable
form.\\

General properties across all tests are: page ranking is always enabled, the
SwingView is always used (it communicates indirect with the model through
the controller using the same method calls as the other views) and the column
'Expected Outcomes' represents the common outcome from all searcher types (in the case of \texttt{LinkedListSearcher}, ignore the number of URLs). 
The outcomes of the tests are found in the Appendices C, D and E and the column
'Actual Outcome' represents the output of all searchers (again, ignore the URL
count in the case of \texttt{LinkedListSearcher}).

\subsubsection{Creating a Test Data-File}
For the purpuse of testing the search functionality (and implicitly the load
of data) we have created a test data-file (test-data.txt). We will create the
data structures from this file during all our black-box tests. It is
characterized by a simplistic structure that allows us to calculate the expected
outcome of any query from knowledge of this structure (we need not look in the
actual file). The structure is as follows:\\

All pages are subpages of the site \texttt{http://www.test.dk}.
Only five different words occur: 'word1', 'word2', 'word3', 'word4',
and 'word5'. When occuring on a page, 'word\texttt{x}' occurs exactly \texttt{x}
times and \texttt{x} will be included in the page URL. Following these simple
rules, the test data-file contains pages covering all possible combinations of
the five different test-words. As an example, the page
\texttt{http://www.test.dk/124} contains the set of words\{ word1, word2,
word2, word4, word4, word4, and word4 \}.\\

Additionally, we constructed two other data-files for testing the loading
process our program against extreme inputs - an empty file (empty.txt) and a
file containing the same words as test-data.txt but with no URLs (no-pages.txt).
The test data sets are found in figure \ref{load_extreme_input}.

\begin{figure}
\begin{center} %centrerer tabellen
 \fontsize{9}{11}\selectfont
\begin{tabular}{|c|c|c|}
\hline
\textbf{Property of file loaded} & \textbf{Query} & \textbf{Expected outcome} \\
\hline
Empty file & word4 & Empty result \\
 
\hline
File containing words but no URLs & word4 & No URL's found \\

\hline
\end{tabular}
\end{center}
\caption{Data load test: Extreme input \label{load_extreme_input}}
\end{figure}

The test results can be found in Appendix C, and they reveal
an unintended outcome when searching for a word that exists within a file
containing no URLs: The user is informed that the word is not found on any URLs
but the mechanism for printing the URL count still outputs a number (this does
not apply to the LinkedListSearcher). Due to the data file not meeting the
expected formatting, we are not sure whether to consider this an error due to
our program relying on specific formatting - i.e., we cannot detect URLs if they
are not part of a string that begins with \texttt{*PAGE:}.

\subsubsection{Test Queries and Expected Outcomes}
First, we will test the search functionality without accounting for
boolean operators. For this purpose, we have constructed the following test
queries (figure \ref{search_without_boolean}). The first two are regular, and
the following are extreme. All outcomes matched our expectations and the results can
be found in Appendix D.\\

\begin{figure}
\begin{center} %centrerer tabellen
 \fontsize{9}{11}\selectfont
\begin{tabular}{|c|c|c|}
\hline
\textbf{Input} & \textbf{Property} & \textbf{Expected outcome} \\
\hline
word3 & exists in data structure & all pages containing word3 \\
\hline
word6 & does not exist in data structure & Empty result \\
\hline
http://www.test.dk/24 & URL existing in test-data.txt & Empty result \\
\hline
*PAGE:http://www.test.dk/24 & URL existing in test-data.txt & Empty result \\
 & formatted as within the file & \\
\hline
\end{tabular}
\end{center}
\caption{Regular and extreme inputs without accounting for boolean
operators\label{search_without_boolean}}
\end{figure}

The primary focus of this test lies in testing searches involving boolean
operators and parentheses. It requires a certain amount of test sets to cover
the many possible combinations of operators and parentheses and some
consideration in choosing a minimal set required for revealing the most probable
errors. The test data sets are found in figure \ref{search_with_boolean}, and
the test results are located in Appendix E.\\

\begin{figure}
\begin{center} %centrerer tabellen
 \fontsize{9}{11}\selectfont
\begin{tabular}{|c|c|c|}
\hline
\textbf{Input} & \textbf{Property} & \textbf{Expected outcome} \\
\hline
word2 & exists in data structure & all pages containing word2 \\
\hline
word6 & does not exist & Empty result \\
 & in datastructure & \\
\hline
(word3) & exists in data structure, & all pages containing word3 \\
 & 1 set of extraneous parentheses & \\
\hline
((((word4)))) & exists in data structure, & all pages containing word4 \\
 & 4 set of extraneous parentheses & \\
\hline
word1 AND word2 & 1 AND operator & all pages satisfying the condition \\
\hline
word3 OR word4 & 1 OR operator & 'same as the above' \\
\hline
word1 AND word3 OR word5 & both AND and OR operators & 'same as the above' \\
\hline
word1 OR word3 AND word5 & the above operators in reverse order & 'same as the
above' \\
\hline
(word1 OR word3) AND word5 & 2 operators and parenthesis & 'same as the
above' \\
\hline
word1 OR (word3 AND word5) & 2 operators and extraneous parenthesis & 'same as the above' \\
\hline
word1 OR (word2 AND & combination of 3 operators & 'same as the above' \\
(word3 OR word4)) & and 2 sets of parentheses & \\
\hline
word1 AND (word2 OR (word3 & combination of 4 operators & 'same as the above' \\
AND word4) OR word5)) & and 3 sets of parentheses & \\
\hline
\end{tabular}
\end{center}\hline
\caption{Regular inputs accounting for boolean operators\label{search_with_boolean}}
\end{figure}

\subsection{Some Final Words on Our Tests}
All outcomes of both white-box and black-box tests
turned out as expected. This is not surprising since we have performed a range of ad-hoc tests during the
development process. On this basis we conclude that our program is likely to
solve its problem properly.
\\
One exeption from this was finding - and subsequently
correcting - an error in the \texttt{readFile}-method of
\texttt{AdvancedGenericSearcher} which testifies to the value of a structured
approach to software testing.



